Analytical evaluation of term weighting schemes for text categorization

1 An analytical evaluation of six widely used term weighting techniques for text cate2 gorization is presented. The analysis depends on expressing the term weights using term 3 occurrence probabilities in positive and negative categories. The weighting behaviors of 4 the schemes considered are firstly clarified by analyzing the relation between the occur5 rence probabilities of terms which receive equal weights. Then, the weights are expressed 6 in terms of ratio and difference of term occurrence probabilities where the similarities and 7 differences among different schemes are revealed. Simulations show that the relative per8 formance of different schemes can be explained by the ways they use ratio and difference 9 of term occurrence probabilities in generating the term weights. 10